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(54) Compensating for backscattered light 

(57) An image of a scene partially obscured by backscattered light Is enhanced by taking account of 
variations m the mean backscattered flux component resulting from variations in the distance between the 
point from which the image Is captured and points in the terrain contributing to the scene. A series of images 
of the scene IS captured, and an averaged image is produced in which each pixel has a brightness which is an 
average of the brightness of those portions of the captured images that represent the same region of the scene 
as that pixel. A model is defined to represent the relationship between the brightness of a pixel of the 
averaged image and the distance between the point the view from which the image represents and the point 
m the scene represented by that pixel of the image. An estimate of the contribution of backscattered light to 
the brightness of each pixel of the image is computed from the model. The estimated contribution for each 
pixel IS then subtracted from the brightness of that pixel of the image to produce a modified brightness for that 
element. The image is reconstructed with each element of the reconstructed image having a brightness which 
IS a function of the modified brightness of that element ^ ^ ^ 
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IMAGE ENH ANCEMEIV T 

The present invention relates to image enhancement, and in particular to a method and 
apparatus for enhancing an image of a scene partially obscured by backscattered light. 

Various techniques are known for enhancing images of scenes which are obscured by 
light backscattered from, for example^ the atmosphere. For example, a camera may be 
mounted on an aircraft to obtain a view of the terrain scene over which that aircraft is flying. 
Assuming that the scene is obscured by low mist, cloud or other atmospheric effects, the 
intensity of light reaching the camera from those terrain features contributing to the scene is 
reduced. A small amount of light scattered from the terrain does reach the camera, but this is 
obscured by light scattered from the mist or cloud. There are many known methods for 
enhancing the contrast of iniages in such circumstances, but the maximum improvement in 
the quality of the image is limited by three factors. 

Firstly, the gain of the camera or other sensing system is set. usually by an automatic 
gain control, to the maximum brightness of the image. When the scattered light component is 
large, the transmitted terrain component becomes small in comparison with the quantisation 
noise of the sensor. 

Secondly, the back scattered light has a random nature, and this is a source of noise 
which is amplified by any contrast-stretching transformation implemented by the sensor. 

Thirdly; in low light conditions, statistical fluctuations in the transmitted photon flux 
give rise to Poisson noise in the image. This noise will be amplified by any transformation 
that increases the range of contrasts present in the image. 

it is known to generate an enhanced image of a scene by averaging information 
related to the scene captured in the form of a series of images representing its appearance 
froni a series of different positions. This technique relies upon prior knowledge of the scene 
in thai the enhanced image is generated by allocating to each pixel of the enhanced image a 
brightness which is the average of the brightness of those ponions of the series of images that 
represent the same part of the scene as that pixel of the enhanced image. This requires 
knowledge of the position, relative to the scene, from which each of the series of images was 
captured and details of the terrain, so that compensation can be made for motion of the 
aircraft as the series of images is generated. Given knowledge of the position from which 
each image of that terrain was captured, and the field of view represented by each of the 



series of images, the images can, in effect, be overlaid in appropriate positions so as to enable 
computation of the average brightness values. A Digital Terrain Elevation (DTE) database is 
already available, together with accurate information about instantaneous aircraft velocity and 
attitude, from on board navigation systems in many modem aircraft. The known motion- 
compensated image averaging systems do provide a significant improvement in image quality 
in conditions of high sensor noise. k 

Enhanced images generated using the known motion-compensated averaging systems 
may be further improved by contrast enhancement. Various contrast enhancement algorithms 
are known, for example variance normalisation or histogram equalisation. In practice 
however such known contrast enhancement algorithms have not provided particularly good 
results. 

It is an object of the present invention to provide an improved method and apparatus 
for enhancing an image of a scene partially obscured by backscattered light. 

According to the present invention, there is provided a method for producing an 
enhanced image of a scene partially obscured by backscattered light, wherein a series of 
images of the scene is captured, an averaged image is produced in which each pixel has a 
brightness which is an average of the brightness of those portions of the captured images that 
represent the same region of the scene as that pixel, a mode! is defined to represent the 
relationship between the brightness of a pixel of the averaged image and the distance between 
the point the view from which the image represents and the point in the scene represented by 
that pixel, an estimate of the contribution of backscattered light to the brightness of each pixel 
of the image is computed from the model, the estimated contribution Tor each pixel is 
subtracted from the brightness of that pixel to produce a modified brightness for that pixel, 
and the enhanced image is formed by allocating lo each pixel of the image a brightness which 
is a function of the modified brightness of that pixel. 

The invention also provides an apparatus for producing an enhanced image of a scene 
partially obscured by backscanered light, comprisinu means for capturing a series of images 
of the scene, means for producing an averaged image in which each pixel has a brightness 
which is an average of the brightness of those portions of the captured images thai represent 
the same region of the scene as that pixel, means for defining a model lo represent the 
relationship between the brightness of a pixel of the a\ eraged image and the distance between 
the point the view from which the image represents and the point in the scene represented by 




thai pixel of the image, means for computing an estimate of the coniribuiion of backscattered 
light to the brightness of each pixel of the image from the model, means for subtracting the 
estimated contribution for each pixel from the brightness of that pixel of the image to produce 
a modified brightness for that pixel, and means for reconstructing the image with each pixel 
of the reconstructed image having a brightness which is a function of the modified brightness 
of that pixel. ^ 

The present invention is based on the realisation that the mean backscattered flux 
component in an image of a scene obscured for example by cloud will var> according to the 
distance between the point from which the image was captured and the points in the terrain 
represented in the image. Depending on the type of view, this distance (or depth) will vary 
across the image. This is particularly the case with aircraft flying generally horizontally 
across the surface of the earth. The invention considers these image-plane variations of depth 
and as a result is robust to variations in mean backscatter. 

The model may be defined by reference to a plot of the depth/brighmess relationship 
for the individual image to be enhanced. 

The brightness of each pixel of the enhanced image may be computed from the 
modified brightness of that pixel scaled to restore the image contrast, for example by 
computing the contribution of light from the scene to the brightness of each pixel of the 
image on the basis of the model, dividing the modified brightness for each pixel by the 
estimated contribution from the scene for that element, and multiplying the resultant by the a 
constant to determine the brightness of the pixel in the enhanced image. 

An embodiment of the present invention will now be described. b\ way of example, 
with reference to the accompanying drawings, in which: 

Figure 1 is a single unprocessed optical image captured by an airborne camera; 

Figure 2 is a histogram of the image of Figure 1 plotting the iniensity against the 
number of pixels in the image having particular inicnsities: 

Figure 3 is an image of the same scene as that represented in Figure ! but representing 
the image resulting iVom the motion-compensated average of ten iVames captured 
sequentially: 

Figure 4 is a histogram corresponding lo thai of Figure 2 but relating to the image of 
Fiyure 3: 
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Figure 5 plots the depth to brightness relationship of the pixels of the image of Figure 

1; 

Figure 6 plots the depth to brightness relationship of the pixels of Figure 3; and 
Figure 7 is an image derived from the image of Figure 3 in accordance with the 
present invention. 

The method of image enhancerftent described below has three steps, that is image 
averaging, parameter estimation, and contrast transformation. Image averaging techniques 
used are conventional, the present invention residing in the combination of these conventional 
techniques with the parameter estimation and contrast transformation steps. 

71ie images described below were obtained from a standard video camera with a field 
of view of 30*^. The camera was mounted on an aircraft of an angle at approximate by 15° to 
the horizontal. Information about the current position, velocity and orientation of the aircraft 
was made available form an on board Inertial Navigation System (INS). A database of terrain 
height values was stored in the onboard computer. Figure 1 represent a single unprocessed 
image generated by the camera. The image is of a scene including a major road, vehicles 
travelling on that road, bridges across the road, and various terrain features to both sides of 
the road. The features of the scene are obscured by low level cloud. A series of ten images 
was taken by the camera as the aircraft travelled relative to the imaged scene. Motion- 
compensated averaging was then applied, the averaging being computed over the ten image 
frames. The image averaging is performed such that the "averaged" image at frame N is 
derived from the sum of a number of previous frames, using a geometric transformation to 
correct for the camera movements. As a result a time-averaged image is mainiained which 
always reflects the current camera position and orientation. Each pixel in this averaged image 
corresponds to some particular point of the terrain which contributes to the iniagc. Assuming 
vvorldspace (terrain) co-ordinates (.\. y. z) this can be wriilcn as the sum: 

where M is the number of images involved in the a\'crauing process, h is \hc input image 
frame k. (xx, y*) is the position of the point {x, y. z) in frame k. 
The averaging process described by equation (1) has twd effects: 

1 . The sensor noise is reduced by a factor of approximately — = 
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2. The statistical fluctuations in the backscattered componeni are also reduced by 
a similar factor. 

Figure 2 is a histogram based on the image of Figure 1 and representing the number of 
pixels in that image having the identified intensities. This histogram is dominated by the 
backscattered light from the cloud. Figure 3 shows the image which results from the motion- 
compensated average of ten frames, and Figure 4 is a histogram corresponding to that of 
Figure 2 but in respect of the image of Figure 3. Figure 4 shows a narrower distribution of 
values around the mean gray level than Figure 2.^ The light flux which has been reflected 
from the terrain is contained within this relatively narrow peak and can now be recovered by 
contrast enhancement. Various contrast enhancement algorithms such as variance 
normalisation or histogram equalisation are suitable for this purpose. One simple algorithm 
which could be used applies the transformation = + c to the image gray levels, with the 
constant m chosen to give good image contrast. In practice however the mean backscattered 
flux component will vary according to .the distance between the camera and the terrain. 
Depending on the type of view, this distance (or depth) will vary across the image. As a 
result the known contrast enhancement algorithms do not result in a very significant 
improvement in the quality of the image represented in Figure 3. The present invention 
however considers these image-plane variations of depth and as a result is robust to variations 
in mean backscatter. 

In accordance with the in\'ention, the backscatter contribution is estimated in each 
pixel in the image. This is achieved b>. considering the depth to brightness relationship of 
many of the pixels in the averaged image. A parametric model is then fitted to the 
dcpth/briuhtness data using a numerical data-fitting algorithm. The resulting model 
parameters are then used to calculate backscatter at any image pixel. This method of 
backscatter estimation also pro\ ides parameters for contrast enhancement. 

Figure 5 plots the depth to brightness relationship of the pixels of the image of Figure 
1 . Figure 6 plots the same data for Figure 3. As might be expected, the motion-compensated 
average iniage of Figure 3 shou's le.ss scatter and therefore represents a better starting point 
for the application of the present invention. 

The steps of parameter estimation and contrast transformation in one implementation 
of the present invention will now be described. 
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An estimate for the backscatter contribution, b = b(d). at each pixel of the image of 
Figure 3 may be obtained from the equation: 

b{d)^-{\-^y^V{-Kd)), (2) 
K 

where / is a constant which depends on the illumination, K is the extinction coefficient which 
characterises the scattering profile of th^ fog/mist/cloud particles, and d is the depth at each 
pixel. Similarly, an estimate for the transmitted terrain contribution at each pixel is given by 

t{d) = Texp{-Kd) (3) 
where 7 is a constant depending on / and the nature of the terrain scene. According to the 
depth-based model, the overall brightness p{d) at a pixel with a depth d is the sum of the 
backscatter b{d) and the light refiected from the terrain r{d), that is 

pid) = bid) + t(d) 

= -(\-txpi-Kd))+Tcxp{-Kd) (4) 
K 

In order to estimate the parameters T, / and /C, a three parameter model was filled to 
the brightness/depth plot of Figure 6. This model is defined by the equation 

p{d) = C0 + c\e\pi-Kd), 

where co = — , and c\ = T — — 
K K 

The model fit is carried out by using a numerical optimisation algorithm to determine 
values for co , co , and AT such that the total least squares difference, defined by 

ulli.i 

where p(i.j) is the brightness of the averaged image ai pixel (/j) and d(i.i) is the 
corresponding depth, is minimised. The dotted line in Figure 6 shows the parametric model. 
The estimated values for co , coand were 217.55, - 145.6.^ and 0.773 respeciivcly. 
The backscatter estimate b(i,j) ai pixel (/,/) is then gi\on by 

b(i../) = ca{\-sxp{-Kd{i.J]) 

and the terrain contribution /(/,;) is given by 

{{i.n = (C0+ c\)sxp[-Kd{i.J)) 
Note that the estimate for takes account of the attenuation of the lerrain-renecied light by 
the scattering medium. 



The enhanced image e{ij) at pixel (/._/') is then generated froni the formula 

e{i.j) = WJ) - b{ij)) (9) 

where b{ij) and t{ij) are calculated using equations (7) and (8) respectively. This is 
equivalent to subtracting the backscatter and scaling to restore full contrast. Note thai the 
images are quantified to 256 gray levels^, The final result is shown in Figure 7. 

A check may be incorporated to ensure that the image data conforms to the model. 
Images in which the extinction coefficient is non-uniform tend to cause failure (the 
production of large or negative pixel values) for large depth values. The algorithm may 
detect any pixels at which the model is violated, and set the gray level of these pixels to the 
maximum value of 255 (i.e white). 

It will be appreciated that alternative methods for estimating the backscattered 
contribution could be used. For example the backscattered contribution could be -estimated 
by applying a low-pass filter to the averaged image. However, this estimate would be 
degraded by any low spatial frequency components in the terrain signals. Also the size of the 
filter kernel would have to be small with respect to the expected variation in depth. This 
would mean that the backscaner estimates from the filter would be subject to a greater degree 
of random uncertainty that with the estimation method described above. 

It will also be appreciated that alternative algorithms could be applied to contrast 
transformation, that is the final generation of the enhanced image from the estimated 
backscatter and transmitted terrain contributions. 
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CLAIMS 

1 . A method for producing an enhanced image of a scene panially obscured by 
backscattered light, wherein a series of images of the scene is captured, an averaged image is 
produced in which each pixel has a brightness which is an average of the brightness of those 
portions of the captured images that represent the same region of the scene as that pixel, a 
model is defined to represent the relationship^ between the brightness of a pixel of the 
averaged image and the distance between the point the view from which the image represents 
and the point in the scene represented by that pixel, an estimate of the contribution of 
backscattered light to the brightness of each pixel of the image is computed from the model, 
the estimated contribution for each pixel is subtracted from the brightness of that pixel to 
produce a modified brightness for that pixel, and the enhanced image is formed by allocating 
to each pixel of the image a brightness which is a function of the modified brightness of that 
pixel. 

2. A method according to claim 1, wherein each element of the enhanced image has a 
brightness computed from the modified brightness of that pixel scaled to restore image 
contrast. 

3. A method according to claim 2, wherein an estimate of the contribmion of light from 
the scene to the brightness of each pixel of the image is computed from the model, and the 
modified brightness for each pixel is divided by the estimated contribution iVom the scene for 
thai pixel and multiplied by a constant to determine the element nrii:hiness in ihe 
reconstructed image. 

4. A method according to any preceding claim, wherein ihc series of imaucs is made up 
from motion-improved images of the scene captured from different positions relative lo the 
scene. 

5. An apparatus for producing an enhanced image of a scene panialK obscured by 
backscattered light, comprising means for capturing a series of images of tlic scene, means for 
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producing an averaged image in which each pixel has a brightness which is an average of the 
brightness of those portions of the captured images that represent the same region of the scene 
as that pixel, means for defining a model to represent the relationship between the brightness 
of a pixel of the averaged image and the distance between the point the view t^om which the 
image represents and the point in the scene represented by that pixel of the image, means for 
computing an estimate of the contribusion of backscattered light to the brightness of each 
pixel of the image from the model, means for subtracting the estimated contribution for each 
pixel from the brightness of that pixel of the image to produce a modified brightness for that 
pixel, and means for reconstructing the image with each pixel of the reconstructed image 
having a brightness which is a function of the modified brightness of that pixel. 

6. A method for producing an enhanced image of a scene partial!) obscured by 
backscattered light substantially as hereinbefore described with reference to the 
accompanying drawings. 



7. An apparatus for enhancing an image of a scene partially obscured by backscattered 
light substantially as hereinbefore described with reference to the accompanying drawings. 
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